Skip to content

Add indexes on serialized_dag and dag_code#69391

Open
justinpakzad wants to merge 1 commit into
apache:mainfrom
justinpakzad:add-indexes-ser-dag-dag-code
Open

Add indexes on serialized_dag and dag_code#69391
justinpakzad wants to merge 1 commit into
apache:mainfrom
justinpakzad:add-indexes-ser-dag-dag-code

Conversation

@justinpakzad

@justinpakzad justinpakzad commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

This PR adds indexes on the dag_id columns of the serialized_dag and dag_code tables. This should avoid full table scans when fetching the latest serialized Dag or Dag code by dag_id.

Note - this a continuation of #64311, which was closed due to inactivity. Thanks to the author for the original implementation.

closes: #60883

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@justinpakzad
justinpakzad marked this pull request as ready for review July 5, 2026 12:26
@justinpakzad
justinpakzad force-pushed the add-indexes-ser-dag-dag-code branch from e897e42 to 95a8040 Compare July 13, 2026 23:42
@justinpakzad

Copy link
Copy Markdown
Contributor Author

Updated the version to 3.3.1. Given there were already a few 3.4.0 migrations, I had to modify/rename some of the other files in order put it between 3.3.0 and 3.4.0. Also added a test as requested.

@justinpakzad
justinpakzad requested a review from ashb July 14, 2026 12:28
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jul 15, 2026
@justinpakzad
justinpakzad force-pushed the add-indexes-ser-dag-dag-code branch from 95a8040 to d2bce90 Compare July 20, 2026 20:52
@justinpakzad

Copy link
Copy Markdown
Contributor Author

Would appreciate another review on this. Thanks.

@justinpakzad
justinpakzad force-pushed the add-indexes-ser-dag-dag-code branch from d2bce90 to f05d856 Compare July 23, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:DAG-processing area:db-migrations PRs with DB migration kind:documentation ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid full table scan on /structure/{dag_id}

4 participants